The automated understanding of simple bar charts

While identifying the intention of an utterance has played a major role in natural language understanding, this work is the first to extend intention recognition to the domain of information graphics. A tenet of this work is the belief that information graphics are a form of language. This is supported by the observation that the overwhelming majority of information graphics from popular media sources appear to have some underlying goal or intended message. As Clark noted, language is more than just words. It is any “signal” (or lack of signal when one is expected), where a signal is a deliberate action that is intended to convey a message [14]. As a form of language, information graphics contain communicative signals that can be used in a computational system to identify the message that the graphic conveys. We identify the communicative signals that appear in simple bar charts, and present an implemented Bayesian network methodology for reasoning about these signals and hypothesizing a graphic’s intended message. Once the message conveyed by an information graphic has been inferred, it can then be used to facilitate access to this information resource for a variety of users, including 1) users of digital libraries, 2) visually impaired users, and 3) users of devices where graphics are impractical or inaccessible.